1. The Field of the Invention
The present invention relates to interprocess communication. More specifically, the present invention relates to methods, systems, and computer program products for high-performance interprocess communication that minimizes context switches, kernel transitions, and blocking.
2. Background and Related Art
Various technologies are available for communicating information from one process to another. Among the most well-known technologies for interprocess communication are remote procedure calls (xe2x80x9cRPCxe2x80x9d), lightweight remote procedure calls (xe2x80x9cLRPCxe2x80x9d), and components based on COM or DCOM. For the most part, well-known interprocess communication techniques are general-purpose solutions that must satisfy a variety of competing interests. One significant drawback to general-purpose solutions is the need to account for the lowest common denominator, foregoing optimizations that might otherwise be available to a particular implementation. As a result, generic interprocess communication often imposes a substantial amount of overhead that tends to degrade throughput and overall performance. Among other things, interprocess communication may increase blocking, kernel transitions, and context switches, with the effect particularly pronounced when two server processes must communicate frequently in achieving a common goal or task.
Because of the higher overhead that interprocess communication is likely to introduce, programs running on a server are often written as monolithic processes to maximize performance. Nevertheless, dividing a particular task into several well-defined processes may provide a substantial benefit in simplifying program design. In choosing between potential improvements in performance versus potential improvements in program design, however, performance frequently is given more weight because end-users invariably detect slow program execution, whereas internal program design is much more difficult for end-users to quantify.
When multiple processes run concurrently on a single processor, the state information that distinguishes one process from another must be kept separate. Switching from one process to another requires storing the state information for the currently executing process and loading the state information for the process to be executed. The storing and loading of state information is known as a context switch. A context switch does not involve executing the instructions of a particular process to achieve a given goal or task; rather, a context switch is merely preparation for executing the instructions of a particular process. The more time spent switching from one process to another, the less time available for running the processes themselves. Of course, some environments may include excess computing resources or sufficient user interaction such that context switches do not represent a significant burden. In other environments, however, especially environments where one process provides one or more services to other processes, context switches may represent a significant amount of processing time that would boost performance if eliminated or reduced.
One way to diminish the impact of context switches is to shrink the amount of state information that must be stored and loaded. So-called lightweight processes or threads are directed largely toward to goal of minimizing state information and the corresponding performance detriments of context switching. Nevertheless, although reduced in extent, whether context switches involve processes or some type of lightweight counterpart, the execution time required for storing and loading context information diminishes performance. Naturally, as the number of competing processes increases, the amount of processor time devoted to context switching may increase as well. Because many of the issues involving interprocess communication are similar for processes, threads, lightweight processes, etc., the term xe2x80x9cprocessxe2x80x9d as used throughout this application should be interpreted as a generic term that encompasses all entities capable execution, including threads, lightweight processes, and the like.
Another factor to consider in evaluating the performance of interprocess communication is the execution time required to marshal data. Data marshaling accounts for system or process differences. For example, the address space of one process may be distinct from the address space of another process, one process may represent data differently from another, etc. Therefore, marshaling generally includes copying parameters, return values, and/or other exchanged data from one process to a shared memory area and then from the shared memory area to another process. As part of the copying, data may be formatted in a portable structure to account for any differences in how the processes represent data. Depending on the data to be exchanged, data marshaling may involve copying, probably multiple times, large amounts of data from one memory location to another. Like context switches, marshaling data is overhead to interprocess communication in which a particular process does not make further progress in achieving its goal or task, but rather, simply prepares for making progress.
At times, one process may need one or more services that another process provides. In using the services of another process, one process often communicates parameters and/or data to the process implementing the needed services and waits for a response. While waiting, the process is xe2x80x9cblocked.xe2x80x9d Blocking is a mixed bag of sorts. A blocked process does not receive any execution time because it is unable to run. By blocking, the process will not be available for a context switch that, if it occurred, would be followed immediately by another context switch because the blocked process cannot continue further processing until it receives a response. Thus, in one sense, blocking saves unproductive context switches. At the same time, however, blocking virtually guarantees one or more context switches. As soon as a process blocks, another process must be selected to run, resulting in one context switch. Then, at a later time when the blocked process receives its response, another context switch occurs to allow the now unblocked process to continue execution.
To illustrate as least some of the foregoing problems, consider the following overview of LRPC. As indicated above, LRPC stands for local remote procedure call and differs from RPC mostly in its underlying implementation. LRPC is used for RPC calls that stay on the same machine, as automated by the RPC subsystem. Rather than network transfer, LRPC uses shared memory for data transfer.
As a specific example, consider LRPC communication between process A and process B. Process A calls into its side of an RPC handler which marshals its data into shared memory, signals process B to handle the call, and then blocks the thread until process B signals that it has completed the request. Process B wakes up when signaled, unmarshals the shared memory data, and calls the registered function with the unmarshaled parameters. Process B waits for the function to return, marshals any return data into shared memory, and then signals process A that it is done. Process A then wakes up, unmarshals the return data, and the RPC routine returns to the caller. Note that LRPC involves multiple context switches, a significant amount of data marshaling, and synchronous interprocess communication that blocks further process execution.
The present invention provides for high-performance interprocess communication without the prohibitive overhead that may be imposed by prior art techniques. During an initialization phase, each side dynamically identifies one or more routines that are responsible for handling communication received from other processes. Communication between processes occurs through a shared memory heap and a shared memory queue. Allocations from the shared memory heap produce a process agnostic memory handle and a process specific memory pointer. Using the memory pointer, the enqueuing process places an operation code, parameters, and any other relevant data in the allocated memory. The enqueuing process then adds the memory handle to the shared memory queue. The dequeuing process uses the memory handle from the shared memory queue to generate a valid memory pointer so that the allocated shared memory can be accessed in the dequeuing process. By allowing direct access to shared memory in each process, the present invention may reduce the amount of data marshaling that otherwise may be required. Upon retrieving the operation code that was placed in the allocated memory, the dequeuing process calls the routine, identified during the initialization phase, that corresponds to the operation code.
When an enqueuing process expects a response back from the dequeuing process, the request may be registered. Upon receiving the response, the enqueuing process removes the request from the register. However, if the dequeuing process is unable to respond, the enqueuing process uses the registration information to free resources allocated to the request and perform any other needed cleanup operations. The dequeuing process may be unable to respond due to a variety of circumstances. For example, the dequeuing process may be shutting down for some reason or the dequeuing process may have experienced a catastrophic failure and been terminated. Thus, depending on the circumstances, the dequeuing process may notify the enqueuing process that it will not respond to any outstanding requests.
The enqueuing and dequeuing operations are asynchronous, thereby allowing multiple enqueues and/or multiple dequeues without a context switch. A signaling arrangement may be used by the enqueuing process to notify the dequeuing process of memory handles that have been added to the shared memory queue. Through the signaling arrangement, the enqueuing process and dequeuing process may help each other in determining the resources that should be devoted to removing memory handles from the shared memory queue and processing the corresponding operation codes. Furthermore, the enqueuing process may reserve space for a response in the shared memory queue. The reserved space assures that the response will not be delayed or cause significant processing overhead due to a fall queue. For example, without reserved space, a response may need to be retried repeatedly until it succeeds or placed on a retry queue which may grow without bound.
In one implementation, the present invention has been used in providing access to content. One process is responsible for content operations and another process is responsible for connection operations. The connection process includes a controller and a bi-directional queue for each supported protocol. A completion port and thread manager pool threads available for dequeuing operations, and a registration controller tracks enqueued communication where a response is expected. Similarly, the store process also includes a controller for each supported protocol, a completion port and thread manager for pooling threads available for dequeuing operations, and a registration controller for tracking enqueued communication where the store process expects a response. Both the connection process and the store process identify handler routines for the operation codes associated with each supported protocol.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the invention. The features and advantages of the invention may be realized and obtained by means of the instruments and combinations particularly pointed out in the appended claims. These and other features of the present invention will become more fully apparent from the following description and appended claims, or may be learned by the practice of the invention as set forth hereinafter.